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Phase II is a continuation of the work initi- 
ated in 1994 with the Project ALERT team. 
Phase it sets out to test the evaluation sur- 
veys developed by the 1994 team and to 
collect feedback on the reactions from both 
tutors and tutees on the structure and ap- 
propriateness of the inquiries. Funded by 
the State Department of Education and 
executed through the Detroit Literacy Coali- 
tion at Wayne State University, the Project 
ALERT team went beyond the confines of 
the metropolitan Detroit area to seek an- 
swers from stakeholders as far away as 
Muskegon, Grand Rapids and Ironwood. 
The hope was to enlist ttie reactions and 
criticisms of many Michiganders in the 
process of reshaping the evaluation tools. 

Daphne W. Ntiri, Associate Professor of 
Social Science, Interdisciplinary Studies 
Program, Wayne State University, who di- 
rected riiis project, is also a State Utemcy 
Facilitator and the Director of the Detroit 
Literacy Coalition, a community-based or- 
ganization in the metro Detroit area. This 
organization meets the needs of at-risk De- 
troiters through the collaborative networking 
of its fifteen member local agencies. Dr. 
Ntiri has written extensively on Adult Edu- 
cation/Literacy in the Third World, especially 
Africa, and, as Co-Director of the Council 



for Excellence ir Adult Learning (CEAL), 
Wayne Suite University, is presently formu- 
lating strategies to advance long-term pro- 
fessional education here in Michigan. 
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The DLC Statement of Purpose 

The purpose of tire Detroit Literacy Coalition (DLC) is to promote the total eradication of illiteracy in our 
community and to advance the cause of reading and writing both for work and pleasure. ITre DLC is 
committed to inproving the qualify of life of every Michigan citizen and resident by ensuring that increased 
access to basic education is available to those deprived or those who for sorrre reason or other missed out 
on early educational opportunities. Its central tenet is that total literacy is fundamental to the well*being of 
Detroit society. 



Program Mission 

With the absence of a sound, scientific and viable instalment to measure effectiveness of statewide vol- 
unteer aduK literacy programs, Project ALERT embarked on the first of a two-part series of an evalua- 
tion model plan to address program evaluation needs. The object was to create a user-friendly evalua- 
tion model that can be applied statewide to volunteer literacy programs. 
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To the Reader 



This is a sequel to an earlier study, 
Evaluation In Adult Utaracy Raa^rch; 
Project ALERT, undertaken in 1994 under 
the auspices of the Detroit Literacy Coalition 
a nonprofit Brganization based at Wayne 
State University, Detroit, Michigan. Eimlua- 
tion In Adult Literacy Reaaareh, Project 
ALEUT, Phase II is dedicated to adult edu- 
cators in administration, instruction and vol- 
unteer sernce who take an interest in ad- 
vancing program quality and effectiveness. 
It has focused on the development of 
evaluation criteria and evaluation instru- 
ments and their application to volunteer lit- 
eracy programs. 

Because of the diversity in adult education 
programming, it seems almost impossible to 
establish a standard evaluation model that 
would be applicable to all programs and 
populations. Given the diversity in provid- 
ers, delivery systems, course offerings, 
support services and adult education learn- 
ers, evaluation outcomes are impacted dif- 
ferently. It is as a result of this complexity 
that the Project ALERT team embarked on 
a mission to fill a void, to create a user- 
friendly model designed to be adaptable 
and flexible, and to provide effective feed- 
back for program improvement and quality. 

The first part of the evaluation research 
study conducted in 1993/4 was the creation 



of a set of evaluation instruments for volun- 
teer adult literacy programs. Modification to 
the instruments led to the completion of a 
set of simplified, time-sensitive and ready- 
to-be-tested questionnaires for tutors and 
tutees in volunteer literacy prograrh. This 
current work continues with the past work 
and is presented in two parts. Phase I pre- 
sents a handbook to literacy administrators 
and other stakeholders on how to construct 
literacy evaluation tools best suited for their 
programs. Phase II shares reactions to a 
set of survey questionnaires tested with tu- 
tors and tutees in the field. The reactions 
were shared on a feedback form. These 
were not about any program per se but 
about the instrument itself, its structure, va- 
lidity, relevance and objectivity. The feed- 
back has assisted witii the generation of a 
refined and scientificstily tested set of in- 
struments. 

The intent of this work remains giving 
evaluation a central place in adult literacy 
programming, which in turn will ensure 
quality assessment and make data avail- 
able for dedsions in program improvement. 



Daphne Williams Ntiri 
Editor & Project Director 
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PREFACE 



The MICHIGAN STATE DEPARTMENT OF 
EDUCATION, OFFICE OF ADULT EX- 
TENDED LEARNING SERVICES con- 
tracted with the DETROIT LITERACY 
COALITION (DLC) to develop and pilot a 
program evaluation model that can be ap- 
plied to literacy operations of community- 
based organizations state-wide under the 
provisions of Section 353 of the Adult Edu- 
cation Act Oftidalty titied ComnnunRy- 
Based Literacy Organization Program 
EvaiuaUon, the project assumed a new 
name during the course of the research: 
AduR Literacy and Research Evaiuation 
Team {ALERT). Because there are several 
other adult literacy non-profit agencies in 
the area wittt a vital interest in the project, 
and because DLC is the key community- 
based Adult Education organization in tiie 
city, the DLC's access to researdi and ex- 
perimentation was significantly enhanced. 

The Detroit Literacy Coalition is a ten-year 
old 501 (c)3 organization that worics vnth key 
literacy non-profit organizations in Detroit to 
bring about collective change and eradicate 
illiteracy. The DLC's network of services 
includes: grants support, staff training and 
development, technical assistance in adult 
literacy resource development and leader- 
ship programs, it serves hundreds of met- 
ropolitan Detroiters at various levels of 
functional illiteracy. The DLC works with 
over sixteen direct literacy providers includ- 
ing the City of Detroit, the Detroit Public Li- 
brary, the Detroit Public Schoois/Adult Edu- 
cation Department, the Archdiocese-sup- 
ported Dominican Literacy Project, Uteracy 
Volunteers of America, AKA Reading pro- 
grams, Wayne State University, Wayne 
County Community College, Highland Park 



Community College and various churches, 
corporations and businesses. The DLC 
gives special attention to sheltered and dis- 
abled populations such as seniors and the 
deaf. A successfully implemented DEAF 
OPTIONS program In 1991 contributed to 
the improvement of literacy education 
among the deaf. 

The organization meets the needs of at-risk 
Detroiters through ttie coliaborative efforts 
of its members and through the over 1,000 
volunteers who serve as instructors, tutors 
and helpers. The Detroit population indudes 
an estimated 200,000 kinctionai illiterates. 
Family and intergenerational illiteracy is 
most common. Other affected populations 
include teenage mothers who contribute to 
the growth of family illiteracy in signiticant 
numbers; the economically depressed who 
have been denied or have denied them- 
selves the opportunities for a sound educa- 
tion; the physically and emotionally chal- 
lenged who have not always been situated 
to benefit from spedal education services; 
masses of dislocated workers suffering from 
recent shifts in global trade and technology. 

The DLC holds community actiwties such 
as the Literacy Summit that assembled lit- 
eracy providers, and supporters to discuss 
and review the problem of illiteracy in this 
sodety and an Annual Walk-A-Thon that 
promotes community consciousness about 
of illiteracy in our communities. The DLC 
hosted the 1992 Annual State Literacy 
Conference that brought over 200 state 
practitioners and theorists together to dis- 
cuss issues and map new trends in Literacy 
and Adult Education. 
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I. INTRODUCTION 



A. Program Goals 

1) To field test a pair of evaluation 
instruments for tutors and tutees 
among various populations in over 
100 volunteer literacy programs 
around the State of Michigan 

2) To obtain feedback on instrument 
structure, content relevance and 
appropriateness through a newly 
designed feedback form 

3) To refine or reconstruct the exist- 
ing models based on data from the 
feedback form 

4) To develop a how-to manual for lit- 
eracy practitioners and administra- 
tors who may not have the knowl- 
edge of developing or applying 
evaluation instruments 

5) To develop a strong research base 
for evaluation of volunteer literacy 
programming 

6) To disseminate the outcomes 
through publications and confer- 
ences 



The Outlook 

Participation and Attrition of Low-Literate '* 
Adults in ABE Programs 

An estimated 18 to 28 million low-literate 
American adults have never entered Adult 
Basic Education (ABE) programs for a 
myriad of reasons (Hunter & Herman, 
1979). Personal barriers such as low self- 



esteem and miscalculating the importance 
and relevance of an education are often 
considered primary explanations for partici- 
pation and attrition irregularities. Though 
ABE information is generally available 
through the public libraries present in every 
city, lack of immediate access to informa- 
tion about ABE programs is often given by 
nonreaders as a reason for nonparticipa- 
tion. 

The actual attrition rate ranges from 40% to 
60% for adult literacy programs, in the 
Farra study (1988), 15% of adults referred 
to a literacy program failed to make contact, 
and 5% failed to enroll after contacting the 
program (Bean, Partanen, Wright & Aaron- 
son, 1989). According to the United States 
Department of Education, the largest of the 
three adult literacy education programs. 
Adult Basic Education (ABE), serves only 
8% of its targeted population. Another 
study of ABE enrollees shows that 35% did 
not complete 12 hours of instruction, and of 
those going beyond 12 hours, 50% com- 
pleted less than 48 hours (Dirkx & Jha, 
1994). 

Knowledge about the systematic differences 
of low-literate adults may require a variety 
of recruitment strategies. Fingeret (1989) 
examined seven assumptions of the Ameri- 
can middle class about nonreading adults: 
dignity, diversity, intelligence, culture, the 
ability to collaborate, aspirations, and resis- 
tance to the educational system. Proven 
research has demonstrated that these as- 
sumptions are biased and lack validity since 
nonliterate learners can be manipulative 
and are known to hold positions of power, 
all the while keeping their illiteracy a secret. 
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B. Program Management 

The grant award was from July 1994 to May 
1995. 

Project ALERT - Phase li was directed by: 

Dr. Daphne W.Ntiri 

Associate Professor, interdisciplinary 
Studies, Wayne Steite University 
Executive Director, Detroit Literacy 
Coalition 

Literacy Fadiitator, Region 4 — State of 
Michigan 



Other members: 

Gregory Robinson 

Graduate Student 

Coundl for Excellence in Adult Learn- 
ing, College of Education, Wayne 
State University 

Gladys Coleman Walden 

Teacher, Detroit Public Sdiools/ABE 

Coundl for Excellence in Adult Learn- 
ing, College of Education, Wayne 
State University 

Weimo Zhu, Ph.D. 

Assistant Professor, College of Educa- 
tion, Wayne State University 



Clerical: 

Carla Jordan 
Crystal Lomax 



Personnel Functions: 

Project Coordination and Presentation 

Dr. Daphne N. Ntiri 



Handbook Development 
Gregory Robinson 

Insiaimentation & implementation 

Glade’S Coleman Walden 

Technical & Statistical Analysis 

Weimo Zhu, Ph.D. 



Sponsored by Office of Adult Extended 
Learning, Michigan Department of 
Education 



EDITOR & PROJECT DIRECTOR 
Project ALERT 

Daphne V\tilliams Ntiri, Ph.D. 

Wayne State University/Detroit Literacy 
Coalition 



ADVISORY BOARD 

Anna Meritt, Director 

Literacy Volunteers of America - Detroit 

Gloria Grady Mills 

Michigan State Department of Education 

Guerin Montilus 
Professor of Anthropology 
Wayne State University 

Sister Marie Schoeniein 
Dominican Literacy Center 

Members of the Advisory Board were consulted 
in different capacities— some for advice on se- 
lection of sites for intenriews, others for input on 
instrument structure, and yet others for careful 
consideration in meeting the state statute. The 
Director and staff would like to acknowledge the 
critical support of the Advisory and regular DLC 
Board in this significant undertaking. 
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EVALUATION HANDBOOK 



for 

ADULT LITERACY PROGRAMS 



Introduction 



Various models of evaluation have been 
reviewed in Adult Education. Such models 
include the predetermined objecth/es ap- 
proach advanced by Tyler (1949); goal-free 
evaluation of Michael Scriven (1972); Kirk- 
patrick’s hierarchy of evaluation (1967), and 
naturalistic evaluation of Egon Guba (1978) 
and Brookfield (1991). That program 
evaluation is central to the ongoing effec- 
tiveness of adult education programs is un- 
questionable. It is a regular and vital seg- 
ment of the adult education discourse be- 
cause of its political Implications, the practi- 
cal realities, and its connection with the 
funding cycle. 




Evaluation data assume significance in the 
decisions of government and public sector 
agencies on the issue of support of adult 
education and literacy programs. Unfortu- 
nately, a “standard” evaluation model for all 
adult literacy programs cannot hold, given 
the varied characteristics of the programs 
and their clientele. According to the litera- 
ture, there is no standard evaluation that 
can be validated as a “model.” Since most 
adult literacy programs are short-term and 
informal, such characteristics tend to be 
handicaps to the evaluation process and 
therefore the subsequent strengthening of 
the program. 

Why are evaluations so relevant and nec- 
essary to the survival of adult education 
programs? Several arguments exist to 
support the relevance and need for program 
evaluation. Reasons include to: 



♦ judge the effectiveness of ongoing 
programs 

♦ seek realistic ways to enhance existing 
programs 

♦ assess the value of innovative pro- 
gram initiatives 

♦ increase the effectiveness of program 
management 

♦ improve cost efficiency 

♦ meet various accountability objectives 

The focus of any educational evaluative 
procedure remains the generation of factual 
data relative to program effectiveness. If 
not conducted competently and scientifi- 
cally, the evaluation outcomes lack meaning 
and can be condemned as worthless. 
Evaluation, therefore, should provide feed- 
back and answer questions that aid in the 
assessment of me^ods used to meet pro- 
gram objectives, and thereby facilitate pro- 
gram improvement. 

In preparation for an evaluation, there 
should be ample discussion of what is to be 
evaluated, and whether internally or exter- 
nally. For both internal or external cases, 
the following elements are necessary: 

♦ the beliefs and expectations of spon- 
sor/employer and evaluator 

♦ the gathering of information 

♦ the formulation of a plan 

♦ the organization of relative goals and 
objectives 

♦ the time frame allotted and budget 
constraints 
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Table 1 

Six Basic Steps for Conducting 
Adult Literacy Program Evaiuation 



1. 


DEVELOP A PRIMARY PLAN OF ACTION 


2. 


SELECT AN EVALUATION INSTRUMENT 


3. 


GATHER RELEVANT DATA 


4. 


COLLECT SUPPORT & FEEDBACK DATA 


5. 


ANALYZE AND PROCESS THE DATA 


6. 


PREPARE RNAL REPORT 



o 
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STEP ONE - Develop a Primary Plan of Action 



Planning is the first important step in devel- 
oping any project. Therefore, the focus of 
the entire planning stage should reflect the 
ideas of stakeholden and the evaluator. 
By focusing on a specific plan, the team can 
narrow objectives and bring greater clarity 
to the evaluation process. Also, by clarify- 
ing goals and objectives, and by sharing all 
relevant information concerning expected 
outcomes with stakeholders, the team vnll 
sharpen its understanding about the total 
process and develop a cohesive approach 
derived from distinct yet related perspec- 
tives. 

Most experts agree that planning a program 
evaluation is a complex task. A handbook 
alone cannot provide everytiiing that is 
needed to know about evaluating a particu- 
lar literacy program. The diversity in volun- 
teer literacy organizations makes it impos- 
sible to recommend an evaluation proce- 
dure for all programs. However, what is of 
note is that the evaluation process must be 
tailored to the particular program. Conse- 
quently, project evaluators agree that the 
following elemente should be incorporated 
into your primary plan of action: 

♦ Plot out ideas and define problems or 
concerns. 

♦ Develop and work out a primary plan 
agenda for discussion. 

♦ Set an initial meeting between evalu- 
ators and sponsors. 

♦ Discuss beliefs and expectations. 

♦ Share relative documentation or data. 

♦ Relate any time, political or budget 
constraints. 

♦ Initiate your evaluation primary plan or 
action. 

♦ 



These points are in keeping with an old but 
wise adage that states: “If you don’t know 
where you’re going, how will you know 
when you get ttiere?" 

Following these steps, the next issue con- 
cerns planning a primary meeting. Please 
note the questions below: 

1) What questions are you trying to an- 
swer from the study, and what are the 
political motives? 

2) What are the resources made avail- 
able to you? 

3) By what system will the data be dis- 
tributed and collected? 

4) How and by whom will the information 
be reported? 

5) How will the results be used? 
(Promote or cut programs?) 

6) How will the instruments be selected? 

7) How much time and money is avaul- 
abie? 

8) Who and by what process will data be 
analyzed? 

Oftentimes, evaluators find that initial ob- 
jectives or specific plans may need chang- 
ing because critical information was lacking. 
So, be prepared for this possibility and be 
guarded against this obstacle by remaining 
objective in your approach. Further, con- 
sider the following options: 

♦ Continue to update and maintain a 
reasonable contact with stakeholders 
or sponsors. 

♦ Set alternate plans during your first 
meeting. 

♦ Know that data collection procedures 
are always subject to a certain degree 
of unreliability. 

♦ incorporate ideas from previously sue- 
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Use common sense. 
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cessful projects in order to modify your 
study. 

♦ Prepare your mind “up-fronf for the 
job. 

Of great significance is "how do evaluators 
manage the diversity of ideas, beliefs and 
expectations that differ between themselves 
and sponsors?” 

The following suggestions may prove use- 
ful: 

1) Remember, there is no such thing as 
too much communication. Don’t as- 
sume anything. ASKI 

2) Make sure everyone understands the 
same definitions that relate to the 
evaluation process (e.g., summative 
vs. formative). 

3) Be ready to explain and define the 
differences between evaluation, as- 
sessment, and measurement proce- 
dures. Although all are related to 
evaluation, they all have distinct func- 
tions and characteristics. 

Defining Evaluation 

Simply defined, evaluation means the 
systematic process of collecting and analyz- 
ing data. L.R. Gay, a noted researcher and 
expert in the field adds, “It is, however, the 
only part of the definition of evaluation on 
which everyone agrees” (p. 6). Studies in- 
dicate that there exists a host of distinct 
differences when defining evaluation. Gay 
further comments, “Examination of evalua- 
tion literature makes it very evident, very 
quickly that there are almost as many defi- 
nitions of evaluations as there are evalua- 
tion experts” (pg. 6). 

He offers these two definitions: 

1 ) Evaluation is the systematic process of 
collecting and analyzing data in order 
to determine whether, and to what de- 
gree, objectives have been, or are 



being, achieved. 

2) Evaluation is the systematic process of 
collecting and analyzing data in order 
to make decisions and improve pro- 
gram outcomes. 

Remember, terms such as assessment, 
measurement and diagnostic, take on dif- 
ferent meanings and functions. See the 
glossary in the back of this handbook for a 
better understanding of these terms. Don’t 
allow the sponsors or other members of the 
team to be confused by these terms. Their 
focus might change and your original objec- 
tives could become altered. 

Based on experiences with the tremendous 
diversity in Michigan’s volunteer adult liter- 
acy programs, ALERT found that the 
“formative” rather than the “summative” ap- 
proach to evaluation worked best for this 
project. 

The ALERT definitions for these terms are 
offered below: 

♦ Formative: An evaluation in which the 
program’s objective is to generate out- 
come data that represent an ongoing 
look at program operation in order to 
improve the program through adjust- 
ment or change. 

♦ Summative: When evaluation results 
contain information given as only a fi- 
nal or definitive assessment of the 
program’s effectiveness and as a ba- 
sis for which funding decisions are 
made. This assessment generally in- 
cludes areas such as student 
achievement, levels of proficiency, 
group scores, test results, perform- 
ance standards, grading scales, etc. 

For additional information and clarity on 
these terms see the table on the following 
page. 



See the Appendix for an ALERT sample 
that provides a listing of agenda items used 
during ALEBTs primary pianning meeting. 
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Table 2 

The Differences Between Summative 
and Formative Evaluations 
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Category 

Comparison 


Formative 

Approach 


Summative 

Approach 


Purpose 


Improvement-oriented 


Certification of program utility 


Key players 


Program staff 
Administration 


Potential sponsors or 
funding agencies 


Evaluation team 
selection: 


Usually internally selected 


Usually externally selected 


Measures 


Ongoing, 
often informal 


Specific time limits 
valid/reliable 


Data collection 
measures 


Frequent, either 
quantitative or 
qualitative 


Limited, usually 
quantitatively designed 


Size of samples 


Often small 


Usually large 


Assessment 

questions 


Defines 

participants/ improvement 
based/examines strengths 
& weaknesses 


Specific outcomes/ 
student achievement test 
scores 


Sample questions 


What is working? 
What needs to be 
improved? 

How do participants feel? 


What results occur? 
With whom? 
Under what conditions? 
At what cost? 



It is strongly recommended that if one’s 
objective is to conduct an evaluation that 
measures such items as student achieve- 
ment, test scores, or other quantitative val- 
ues, a “summative” framework be used af- 
ter the completion of a semester. . 

On the other hand, if one’s objective is to 
demonstrate that an ongoing program is 
worthwhile and the main concern is to 
evaluate the operational procedures them- 
selves, then a “formative” approach might 
prove more useful. 

Below are a few more hints to consider be- 
fore going on to Step Two. Double check to 
be sure: 



♦ that all involved parties and/or sponsors 
are informed about exactly when the 
project will start and what agencies and 
populations will be surveyed 

♦ that you know how each member of the 
team feels about the process 

♦ that you determine and agree on each 
team member’s role in the process 

♦ that you specify the length of the entire 
project, and schedule additional meeting 
dates for the planning committee and 

♦ that you prepare a cover letter to ac- 
company your evaluation that is clear in 
stated goals and mission for the project. 



Summary: Developing a Primary Plan of Action 
(Step One) 



1 . Brainstorm to plot out ideas and define problem areas. 

2. Set a meeting date between evaluators and sponsors. 

3. Write an agenda to discuss goals and objectives, beliefs and expecta- 
tions. 

4. Share any related data or documentation. 

5. Relate all time, political, or budgetary constraints. 

6. Formulate and initiate the evaluation plan of action. 
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STEP TWO: Selecting Evaluation Instruments 



Determining what evaluation instrument is 
best suited for your particular adult literacy 
program is very critical. The instrument 
selected slK>uld be capable of evaluating 
the oven^l program objectives. 

In general, it is as convenient to use a stan- 
dardized instrument that has been designed 
to evaluate similar programs as to construct 
your own. It is a good idea to find out what 
evaluation resources are available before 
deciding on a tool or initiating the instrument 
development process. Gay (1980) notes. 
"There are situations, however, for which 
use of available instruments is impractical, 
ins 4 )propriate, or both” (p. 110), so be pre- 
pared to add survey questions to a stan- 
dardized form or design a totally different 
form altogether. 

As we mentioned in the previous section, 
"formative” approaches work best for 
evaluating operational procedures. It is im- 
possbie to evaluate every single compo- 
nent of a literacy program, as there are 



too many conaderations. The aim and in- 
tent should be to narrow the variables and 
determine which elements are the most im- 
portant in meeting assessment objectives. 
Therefore, the instrument should reflect 
those goals and objectives set during the 
primary planning stage. 

Keep in mind that you don’t want to over- 
whelm respondents by making question- 
naires or interview questions too long. Try. 
as much as possible, to limit sections of the 
questionnaires to 10-15 questions, if the 
questionnaire requires answers for different 
categories, use no more than tfrree sec- 
tions. Some of the general areas you might 
want to focus on include: Who is served by 
the program? How were they chosen to 
participate? How is the program funded? 
What type of services are offered? 

To determine if the instrument will enhance 
a formative evaluation procedure, consider 
these 10 major pointe: 



1. Does it identify stixtefit ne^ 

2. Does it evaluate any addifibr^ ser>rices offered (e.g.. couns^ing)? 

3. Does it evaluate whether ^e dittk^ te meetii^ Stude^ O^ls? 

4. Does ttevcduBitei the 

5. Does it evaluate if leai^ng supines or tee suited text a^lte)to7 

6. Does it evaluate wtietiter tutc^ 

7. Does hevteuate the tedem to which learrrii^g is achieved? 

8. Does it evaluate what outside barriers mi(^tpie\tent students from meeting goals? 

9. Does it evaluate how the program has enhanced or promoted growth and opportunity in 
students* lives? 

10. Does it evaluate the specific needs or rru^ strengths of your program? 
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Most of the 10 elements mentioned above 
are included in the ALERT models. ALERT 
designed ^ree different instruments in an 
effort to cover these critical areas. In most 
evaluation processes, only one of ttisse in- 
struments is needed. The first model is a 
very lengthy form designed to address a 
spectrum of literacy program concerns, it is 
complete with questionnaires for evaluation 
of administrators, tutors, and tutees. The 
second model is a shorter version of ttie 
first, and does not evaluate administrative 
staff. The last revision, or third model, is 
totally different from the first two, and it par- 
ticularly addresses the feedback concerns 
solicited from over 100 literacy providers in 
the state. 

A review of ail three ALERT models 



(examples of the first two models are pro- 
vided in the Appendix) is recommended 
before deciding which tool might work best 
for your particular program. The first model 
serves as a prototype for the other two. It is 
comprehensive and designed for lengthy 
evaluation projects. The shorter revised 
models are less formative and are time- 
sensitive, but also cover many of the most 
critical areas. 

A full sample of the third model is included 
in the Appendix. 

The following criteria might further assist 
you in determining what type of instrument 
will work best for your particular evaluation 
project. 



1. Does the instrument significantly increase your imowiedge and understanding of tiie ele- 
ments to be investig|ated? 

2. Can the infoniMition from the questionnaire be categorically or ^stematicadly used to en- 
hance the analysis prpcess in the nr^ efficient smd timely teshion? 

3. Can the date coNe^ed best be used in a qualitative <x quantitative manner (this should be 
decided before the actual evaluation procedures begin)? 

4. Determine witichinstriteveiri is ec^st to ur^rstand and use. is it time-sensitive and user- 
friendly, espedaHy the questions desigried for the atudents? 

5. WHi the infonnation and knowledge enhance eventual understanding about the overall pro- 
gram? Could this lead to better dectejonrmaidng practices? 

6. Please remember that if the standardized instrument is not appropriate for your particular 
iiteia^ program, then it may become necessary to design or dmrelop your own. 
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If this does become the case, keep these 
hints in mind: 

1. Because of the uniqueness of this 
handbook, the infomnation provided 
can be used in almost any literacy 
evaluation project, usually with very 
few alterations. 

2. A good practice is to include the ideas, 
suggestions and reflections of staff, 
sponsors and students to help vnth 
tool development. 

3. Don't be afraid to utilize the mass 
amount of literature on literacy re- 
search and evaluation procedures 
published by experts in the field. 



4. As the primary evaluator, remember 
that your opinions are valuable. Fol- 
low basic guidelines, but use common 
sense and experience because you 
must feel comfortable vnth what you 
are doing. 

Before you go to the next step, please pilot- 
test the instrument for validity and reliability. 
Contact a literacy provider in your group or 
area by sending an announcement letter 
and follow up with a phone call. Expisun 
that he or she was chosen to pilot-test the 
instrument, and then send a sample of tfie 
questionnaire, requesting a speedy return. 
Be sure to ask some open-ended questions 
as well. 



Summary: Selecting Evaluation instruments 

(Step Two) 



1. Brainstorm to determine if ttie study will be quaiitative or quantitative in nature. 

2. Seek a standardized instrument that could best evaluate the progtam’e ot^ectives. 

3. Com^te the starkfaud instrurneht yourself to rnake ixjfd it is time*8ensitive and user- 
hientUy. . 

4. Be prepared to make acQustments to the standard form or to de^ a separate form aito- 
gettrer. 

5. Design and send out a letter to a selected litercu^ provider asking for assistance to pilot-test 
the jnsbuiheriL 

6. Upon ri^m of the pre-t^ instrurnent, mview the fl and share responses witii stake- 
holders (h* sporisors. 

7. Me^e any nec^sary adjustments to the instrument or to procedures. 
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STEP THREE: Gathering Relevant Data 



The third step in an evaluation project is 
gathering relevant data. ITiis process en- 
tails how one goes about collecting the in- 
formation surveyed. An evaluation is only 
as good as the information gathered. A 
common weakness in many evaluation 
projects is the focus of too much attention 
on too many areas by evaluators with too 
little time and limited information. Don’t 
make tiiis mistake! Your results are likely to 
be invalid. The process of gathering and 
collecting the most useful and relevant data 
becomes critical. 

This step serves as a conceptual menu that 
directs the evaluator toward specific cate- 
gories of relevant data to enhance the col- 
lection and analysis process, in this proc- 
ess, it is a good idea to compile a listing of 
ail potential literacy providers you want to 
survey. Before you send out your ques- 
tionnaire, you must first send out a short, 
direct cover letter. Some main points to be 
addressed are: the name of the organize’ 
tion conducting the study: the reason for the 
study: some very brief instructions: a return 
address and a deadiine date: and a thank- 
you in advance for participating. (A sampie 
of the ALERT cover ietter is provided in the 
Appendix.) 

Once you have allowed enough time for re- 
spondents to receive the cover ietter, it is a 
good idea to make a follow-up phone call to 
ensure that the ietter was received and to 
answer any questions the recipient might 
have. 

Now you are ready to send out your ques- 
tionnsures. A questionnaire is Just one of 
many methods of gathering information, but 
most experts agree that it is probably the 
best method. The questionnaire survey is 
frequently used as a polling method to 
gather facts, it is very important that you 



send the questionnaire out in a timely fash- 
ion, allowing program facilitators adequate 
opportunity to respond by your return date. 

A good evaluator will always respect pri- 
vacy, confidentiality, and the rights of tiie 
indhriduai being surveyed. To assure re- 
spondents that your study is etiiicai and will 
be conducted with sensitivity and integrity, 
explain the procedure to tutors and students 
during interviews, or with program facilita- 
tors if interviewing will not be done. Always 
ask for permission to use data that might be 
considered confidential. Remember that 
evaluations can be extremely provocative 
and even threatening, if a potential re- 
spondent feels threatened, he/she might not 
respond with honesty— or might not re- 
spond at ail. 

Be sure that the directions request a realis- 
tic return date, and that it is clear and easy 
to find in the package. Note also that rarely 
are 100% of the questionnaires distributed 
returned, in fact, in most cases, only about 
60% of potential respondents respond after 
the first contact. To assure that a greater 
percentage of samples eventually will be 
returned, be sure to: 

♦ allow additional time for late responses 

♦ prepare to visit some sites 

♦ send out a second cover letter 

♦ make additional follow-up calls 

When a follow-up plan is in place, usually 
another 20-30% of samples are returned. 
Efficiency, then, is a very important element 
in the process. A balance between avail- 
able time to collect the data, cost require- 
ments, and coding and organizing results 
must be considered. 

There are a number of methods to gather 
information and collect data. Please decide 
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early on which techniques will woiic best for 
your particular situation. Keep in mind 
tiiese constrsunts: budget, schedule, and 
availability and reliability of resources. Re- 
liability here represents the elements in- 
volved in the judgment by sponsors in such 
areas as the items being evaluated, the ex- 
aminers, the examinees, and die accuracy 
of outcomes. 

Patterns of a reliable study can take on 
three different dimensions: objective, direct. 



and structural. The objective approach in- 
volves an implicit frame of reference where 
the concepts of accuracy and error are both 
considered, in the direct approach, re- 
spondents and data collectors agree on the 
mettiod and purpose of the collection proc- 
ess. in a structural framewoilc, however, 
specific lands of responses are determined 
relevant. 



Summary: Gathering Relevant Data 

(Step Three) 



1. compile a list dfadiiitera^ 

2. Sehd eabh prayi^ who you are and your intentions. 

3. Make fofloW-up phdne received, and answer questions. 

4. Serid <xit dre in^ment or start ihteni^winng respoiKients. 

5. Have a lotidw-t|p; ki jpiace to assure that the maidmum number of responses are re- 
turned aixl cotiec^ 

6. Immediately oodHy pf organize responses as tiiey are returned. 

7. After the deacRine date has «^'red. prepare to send all data to die analysis expert 
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STEP FOUR: Soliciting Support and Feedback Data 



Once the cpiiection of data and data analy- 
sis process are complete, fine-tuning of ef- 
forts is needed to test the variations among 
diverse target populations. For example, a 
standardized instrument that worked fine for 
evaluating an outreach literacy center may 
have little relevance, if any, when evaluating 
prison populations or workplace literacy 
programs. 

in other words, because of differences in 
target groups in literacy programs, a second 
revision of the instrument may prove nec- 
essary. This is often not required until the 
primary evaluation process has been com- 
pleted. The second revision assures stake- 
holders that a fair assessment of these 
groups can be implemented, and that, in so 
doing, evaluation services delivered to the 
general literacy population are equal to 
services delivered to the special targeted 
population. 

Remember, however, that no evaluation 
tool can cover all of the assessment and 
measurement needs of so many different 
literacy providers. In many cases, once a 
feedback questionnaire is circulated among 
some of the targeted populations, it can 
provide substantive data that can add to the 
validity of a survey without complicating 
procedures in great deal. 



There are several techniques that one can 
use to gain greater feedback from the tar- 
geted groups. However, a good evaluator 
will narrow the choices in order to stay on 
pace viflth the teams’ objectives. The bases 
for implementing such changes usually fail 
on the need to provide more impressionistic 
evidence, because certain literacy providers 
or fadlitators will be dissatisfied with the 
way efforts were conducted in relation to 
their particular program. A feedback ap- 
proach is less threatening and more inviting. 

The following elements should be consid- 
ered before attempting the support and 
feedbadc process: 

♦ How to identify where target problems 
and populations exist. 

♦ What types of procedures or sen/ices 
to provide to target groups. 

♦ How to assess the qualifications and 
competencies of staff. 

♦ How to encourage cooperation from 
the target program’s staff and stu- 
dents. 

♦ How to ensure strict ccnfidentiality in 
order to maintain reliability and validity. 

♦ How to use new evidence. 
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Summary: Soliciting Support and Feedback Data 

(Step Four) 



1. Based on the responses of the initiai survey, determine which programs can add greater 
validity to the Study if questions are more generaiiy designed to address their needs. Also, 
iden%:pro 9 ^s that did iK)t respond or are omitted in toe first study because they serv- 
iced a spectai group. 

2. Design and send out another cover letter along with a copy of toe first questionnaire to 
these groups, asking them for their input 

3. As soon as responses are returned, outline and coliate the feedback data where appro- 
priate;, ..-'y 

4. BirTMtiato urateoe^ feedback, arto determine whto^ responses could en- 

hanrte toe proc^ 

5. Use toe relent feedback data to revise toe original questionnaire. 
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STEP FIVE: Processing, Analyzing, 
and Summarizing Data 



The data analysis, or interpretation of re- 
sults, is i^tai to the evaluation process. A 
well-developed analysis leads directly to the 
formulation of conclusions that can impact 
dedsion-making practices. Accurate statis- 
tical interpretation, tharefore, plays a major 
role in the analysis process as well. The 
statistics in evaluative research involve the 
process of describing, synthesizing and in- 
terpreting the quantitative data. There are a 
number of ways to analyze data. Determin- 
ing which techniques vwll work for your par- 
ticular study greatly depends on such fac- 
tors as type of evaluation, evaluation ques- 
tions, and design of the evaluation instru- 
ment. To avoid conflicts in interpretation of 
the data, arrange meetings with the statisti- 
cal analyst. 

The procedure that you select for analyzing 
the data will differ, depending on whether 
the information collected is quantitative or 
qualitative. In general, a quantitative analy- 
sis measures the extent that data is re- 
ported numerically, while qualitative data is, 
in most cases, gathered ttirough open- 
ended answers and is value-based (see the 
glossary for a better definition of these 
terms). 

A good analyst will, upon completion of 
collecting ail data, select and organize the 
information according to the evaluation 
questions. Also, if appropriate, he/she will 
collate into specific categories the data ob- 



tained from each source or method. For 
example, all the data will be coded to reflect 
differences in gender, age between tutors 
and students. 

if the members of your evaluation staff are 
not qualified to prepare the analysis, then it 
is better to hire someone who knows first- 
hand how to handle the process. If this is 
not possible, be sure that the evaluator se- 
lected for the job is well-trained and has the 
time and patience to do the job. 

The analysis should be constructed and 
presented in a way that addresses the proj- 
ect’s evaluation objectives. The ALERT 
project, for example, was quantitative in 
design and formative in nature; therefore, 
all statistical evidence was taken and col- 
lated to measure the quality of the ongoing 
programming and to determine the signifi- 
cance of the outcomes provided. In order 
to achieve these ends, it was necessary 
that the data given in this stage was pre- 
sented simply, accurately, scientifically, and 
clearly. It is also a good idea to identify ail 
themes that might be suggested by the 
findings. Also, wherever possible, provide 
charts, tables, and graphs to express data 
from different lists or counts in order to aid 
sponsors and stakeholders to understand 
and interpret the information. Review tiie 
bibliography of this handbook for more re- 
source assistance. 
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Summary: Processing, Analyzing, and Summarizing Data 
(Step Five) 



1 . Select a welMrained staff member for the task, or hire an analysis expert 

2. Select and organize the data according to the evaluation questions and the types of or- 
ganizations responding. 

3. Wherever needed, make sure the infomnation is collated into specific sub-groups. 

4. Select tile educationeti;aiiaIy^ cpmpr^r p^ tiiat best meets your needs. 

5. Make sui^ that pndel ^^^ia^^ the information is dear, easy to 

:'undei8t8ufkli:ai^ 

6. Provide chaste, t^es, a^^ graphs to hdp validate and express the findings to stake- 
■. /.hokiere■or«the^1^^dnsors,;^/■^y■. : ^ 

7. Pr^re the final ahaiyi^ so it can be incorporated into the final report 
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STEP SIX: The Final Report 



The last step in an adult literacy evaluation 
is the final report. The writing of the report 
should remain as objective as possible. 
The report should not contain subjective 
opinions, remarks, or statements. There is 
dso no place for emotionalism in the report. 
As much as possible, stay away from per- 
sonal pronouns such as “I," “my," and “we." 
For example, instead of saying, “We ran- 
domly selected samples," the correct ex- 
pression would be, “The sample was ran- 
domly selected." 

The report should include information or a 
summary from each step of the project. But 
it should be written in a straightforward style 
containing clear and simple language. 
Make sure that the information presented in 
the final report is consistent with the facts 
illustrated in the study and with table, charts 
and graphs representing the findings. Addi- 
tional comments should be made (as briefly 
as possible) for every instrument or re- 
search tool that may have been used in the 
study. 



O 




A section describing the methods used in 
the study is always an excellent addition to 
any evaluation report. When describing the 
methodology, include in your statement 
comments concerning the targeted popula- 
tions, the measuring instrument used, pre- 
plan statements, the actual evaluation de- 
sign, specific procedures followed, and any 
limitation, biases or probiems that might 
have occurred. 

Stakeholders or sponsors should be made 
aware of all findings and recommendations 
before results are published. Where appli- 
cable, their suggestions and interpretations 
of the findings should be included in the 
summaiy. Remember, their involvement 
can add credence to the study and will pro- 
vide them a sense of teamwork as a mem- 
ber of the research group. 

Rnaliy, be sure to detail all relative recom- 
mendations and conclusions found. The 
recommendations in any evaluation report 
take the form of suggestions concerning 
ways to improve the program or the evalua- 
tion project itself. 
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Summary: The Final Report 
(Step Six) 

1 . Review all data from every step in the process and summarize the findings. 

2. Share the findings and your recommendations with stakeholders or sponsors. 

3. Use a computer (or other word processor) to type all frie data. 

4. Allow at least two people to proofread the final report before it is published. 

5. Make sure that the repc^ contains all ttie^ oorn^^ 

A. An infroducfk^, :ma^^ goals and mission of the project 

B. A methodoipgy sec^ describing the questionnaire process and the 
targeted poptda|^ 

C. Data analysis of results (e^g., graphs; t^ 

D. All recommendations, conclusions and summaries. 
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Table 3 

Evaluation Methods Chart 


PROCEDURES 


ADVANTAGES 


DISADVANTAGES 


GROUP INTERVIEWS 


Stimulates thinking and sharing of ideas and 
experiences. 

Provides a greater range for diversity pertain- 
ing to the subject. 

Provides a consensus about the program. 


Limits confidantlAtity. 

Is more difficult to organize. 

Can be considered as a threatening environ- 
ment 

May require a more skilled or particularly trained 
interviewer. 


MOIVKXIAL 

INTERVIEWS 


More likely to allow for sincere responses. 

May make participants more willing to dis- 
close personal information. 

Allows interviewers to control how and when 
Questions are asked. 


Usually is more time-consuming. 

Is generally more difficult to analyze. 

Will require a skilled Interviewer. 

Participants can sometimes feel threatened. 


QUESTIONNAIRE 

SURVEYS 


Large amounts of data can be collected in 
short periods of lime. 

Is usually less expensive than other forms of 
surveying. 

Is better for maintaining confidentiality. 

Makes the task of analyzing and summarizing 
data less dHficutt. 


Data are restricted by the way certain questions 
are asked. 

Greater planning time Is needed. 

Return rates are usually slower coming in. 


QUANTITATIVE 

EVALUATION 


Allows for mors sampiss, which adds mors 
asdsncs to ths survsy. 

Dsals «^<h spscific numbers. 

Provides better information If results are pre- 
sented in table or graph form. 

Data ara usually aasiar to collact, and is done 
in shorter periods of time. 

Is generally aasiar to summarize arxi analyze 
data. 


Does not allow participants to respond objec- 
tively. 

Requires a skilled quantitative analysis expert. 

Provides greater limitations in the amount of 
information that can be collected and analyzed. 

Can be seen as more threatening. 


QUALITATIVE 

EVALUATION 


Allowe for participants to respond objectively 
and freely. 

Allows for open-ended responses, which add 
direct credibility to the study. 

Intsnriswsrs can get different personal views 
on same subject 

Can control when and how questions are 

asked. 

Allows irrtstviewsis to gain a bstttr interpreta- 
tion as to ths wav rasDondents actually fast. 


Substantial planning dm# is raquirsd. 

Rstum rates ars usually vary low. 

Participanta may not be dirsetiy familiar with 
program. 

la more difficult to analyze. 

Is limited in confidentiality. 

May ba difficult to organize. 

Uauatly reflects quality outcomes only. 


JOURNALS/LOGS 


Offers immediate reaction to svsnta or activ- 
ity. 

Requires IKtls effort to oollsct data. 

Is lass expensive to administer. 

Offers records chanoe over time. 


Data ara unreliable, end more subjective. 

Are more difficult to analyze. 

More training may be required in order to record 
information. 
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METHODOLOGY 

This second phase of Project ALERT set 
out to collect reactions to the evaluation 
model (see ^pendix). The prototype de- 
veloped in the fimt phase was created 
through interviews with administrators, staff 
and students at primary literacy providers 
(Detroit Public ^hoois. Adult Education 
Division; Literacy Volunteers of America; 
Dominican Literacy Center) to ensure that 
the critical priority areas of volunteer literacy 
program instruction and administration were 
covered, that is. attendance, personal 
goals, counseling and motivation notwith- 
standing. 

To maximize the share of reactions on the 
sunreys. an instrument was designed and 
called a feedbadc form (see Appendix). 
The first page of tiie feedbadc instrument 
collected data on both the program and the 
respondent. It was categorized into the fol- 
lowing: 

♦ area setting of the program 

♦ years in adult literacy 

♦ level of education 

♦ state teacher certification 

♦ gender 

♦ ethnic/raciai composition 

The second page solidted information 
about the student and tutor evaluation sur- 
vey questionnaire \mth regard to length and 
structure of the instrument, appropriateness 
of questions, clarity and applicability of 
questionnaire to respective program popu- 
lations. 



Instrumentation 

Evaluation in Adult Literacy Research: 
Project ALERT- Phase il deals with three 
separate survey instiximents which have 
been discussed: 

1. A Self-Evaluation Program Tutor Sur- 
vey Questionnaire 

2. A Self-Evaluation Program Student 
Survey Questionnaire 

3. A Feedback Form 
Population 

A set of ail three instruments mentioned in 
instrumentation, accompanied by a self- 
addressed envelope was sent to 221 
(N=221) literacy program partidpants 
throughout Michigan before November 14. 
1995. Listings were pro\tided by the Michi- 
gan Literacy. Inc. directory (134 adult liter- 
acy organization) and the State Action Plan 
Team (25 Michigan student support 
groups). Respondents were asked to study 
the survey questionnaires and answer 
questions on ^e feedback instrument vtith 
all their comments no later than December 
31, 1994. A total of 64 subjects (30%) re- 
turned the survey. 

Revised Student end Tutor Seif- 
Evaluation Program Questionnaire 

The responses received allowed for the 
creation of revised student and tutor pro- 
gram survey questionnaires, each of which 
was divided into three parts: 

Part I identification 

Part II Program Information 
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Part III Program Issues 

All three parts of the revised evaluation 
models can be given in segments. Part I 
can be given when the participants first en- 
ter the program. Part II is designed to be 
answered after participants have been in 
the program for a minimum of two instruc- 
tional sessions. Part ill was developed for 
participants who have a minimum of four 
instructional sessions. 

it is advisable that the tutor’s re>nsed 
evaluation be completed at one time or in 
segments. However, the student’s revised 
evaluation model should preferably be 
completed in segmente over a short period 
of time because of reading aind attention 
levels, and to minimize anxiety and frustra- 
tion. 

Feedback Comments 

The feedback comments are extensive and 
reflect common concerns of adult educators 
in the field on designing a model evaluation 
instrument. They have been used to re- 
shape the outlook of the original evaluation 
model. Four areas of concern are noted: 

♦ tutee/student readability 

♦ word usage 

♦ relevance and applicability of ques- 
tions in the tutor questionnaire to vol- 
unteer literacy programs 

♦ questions for tutors on tutees’ thinking 
patterns and perceptions 

Tut0e fteadabtilty/Word U»Mge 

The evidence shows that the tutee/student 
had difficulties of comprehension of the in- 
struments. A significant number of them 
thought that the student evaluation form 
was not ”new reader” friendly and therefore 
comprehension was limited, from length 
was frightening and language was difficult. 
For example, some of the tutor’s comments 



showed preferences over the use of certain 
words: 

Use the word “answer” instead of 
“response” 

Use “helped to learn” instead of “gave you 
information” 

Use “signing up” instead of “enroiimenf 
Use the word “tutor” instead of “instructor” 

The tutors shared that tutees would be un- 
able to complete the questionnaire wthout 
assistance, and for ffiis responding tutor, 
the tutees she was in contact with found the 
form “scary,” “hard to read” and “it made 
them nenrous.” She continued that one of 
the students reading the word “reservation,” 
meaning restraint, confused this term vrith 
an Indian reservation in the next county. 
Equally, words like ‘purpose,’ ‘response,’ 
‘sources,’ ‘referral.’ are at a higher reading 
level and thus cannot be fathomed by new 
readers. 

On the issue of word usage in the ques- 
tionnaire, there were concerns related to 
ambiguity and difficulty levels. The com- 
mentaries pointed to difficulties in distin- 
guishing differences in words and shades of 
meaning. 

What is the difference between “to a great 
extent" and “to a good extent" 

What is the difference between “Just righf 
and “challenging" 

Words like “purpose," “response" are 
complex wjrds for real beginners. Yes, 
the tutor becomes the reader for the stu- 
dent depending on ffte level of difficulty. 

What about a category for “don’t know" 

Tutors’ Speaking for Tutees 

The tutors were quite candid and forthright 
about the shortcomings of the instruments. 
Though fully cognizant of the limitations of 




the tutees/adult learners with limited reading 
and writing skills, they (tutors) were op- 
posed to being asked about '^^rhat the 
tutees/students think or what the tutees/ 
students perceive about their learning proc- 
esses. The reasoning behind their opposi- 
tion touched on factors that relate to validat- 
ing perceptions of others who are less for- 
tunate. Further, they complained about the 
instrument’s failure to address tutor training 
needs and the volunteer nature of employ- 
ment for many of the tutors. Some added 
that some questions bordered on irrele- 
vance and inappropriateness. The word 
“center" was used to refer to literacy pro- 
grams, but in essence not all literacy pro- 
grams see themselves as, or call them- 
selves centers. The reactions that follow 
articulate more directly and restate some of 
the questions as they should have been 
presented: 

— The instrument asks nothing about 
support or training needs for the tutors 

/ wouid personaiiy iike to see more 
questions for the tutor regarding what 
kind of feedback they wouid iike from 
the organization that they do not get. i 
wouid iike to see more questions 
geared toward improving retention and 
making it a positive experience for the 
tutor 

— Some of these questions have nothing 
to do with tutor satisfaction or prob- 
iems/concems. it seems to be more of 
a vaiidation of the student question- 
naire 

— How about these types of questions 
instead of the ones given 

o average number of times student 
canceis/misses a session 

o average number of times tutor 
canceis/misses a session 

o average number of sessions 
scheduied during month 



On the question of whether the tutee rec- 
ommends the program to others or about 
the perceptions of tutees by tutors 
(questions 9 and 13), many expressed 
frustration and resentment. They noted; 

— many of the questions (9, 12, 13, 14, 
16, 17, 18) do not ask tutors what they 
think but rather they ask the tutors what 
the students think or what the students 
perceive, is it fair, on a formai ques- 
tionnaire, to ask any person what an- 
other person thinks? 

— our tutors cannot generaiize about stu- 
dents in the program because they 
know oniy their own student weii. 

— some of the questions (9, 10, 11, 12, 
14, 17, 18) are written in the piurai 
form. Since most tutors in the state are 
trained to work with oniy one student at 
a time, they might feel this is a ques- 
tionnaire for teachers, not tutors. 

— some questions such as opinion of stu- 
dent participant cannot be answered 
coiiec^veiy. What is the purpose of 
asking questions reiating to students 
without asking them? That’s difficuit 
and impracdcai. 

On the question, “please indicate the extent 
to which “your” student expectations are 
being met in the achievement of their goal,” 
one respondent would rather have this 
question asked: 

What is the extent to which program/staff 
are meeting tutor's expectation/need for 
support? 

The instrument elicited responses that ex- 
posed deep-seated limitations about the 
adult literacy volunteer sector. The senti- 
ments are strong on the part of the tutors 
on expanding tutor training and enhancing 
classroom techniques to upgrade results. 
One of the main objectives of evaluation is 
to generate concrete data that will improve 
services and outcomes. Questions that 
surface in this inquiry are: Did we learn from 
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this survey? How will the data be used to 
effect outcomes? What ottier questions are 
outstanding that need to be included? 
Should more time have been spent on ask- 
ing respondents about instructional packag- 
ing and delivery in the ciassroom, or ciass- 
room interactions between tutor and tutee? 
Should ttiere have been a focus on the im- 
pact of technoiogy on adult education’s 
ciassroom? How about support services 
and its links with student performance? 

As mentioned in the beginning, there is 
nothing like a “standard” evaluation that can 
be uniformly applied to all programs. The 
intent of this field exercise was to send out 
sensors, get resuite and interpret them as 
part of a mechanism toward program im- 
provement. 
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FEEDBACK ON EVALUATION QUESTIONNAIRE 

PART II 



PROCESS 

Survey Instrument Development 

A survey instrument (see Appendix) was 
developed for this project. The instrument 
consisted of two major parts: respondents’ 
demographics and their feedback on the 
evaluation questionnaires. There were six 
questions in the first part focusing on 
agency personnel background and the 
characteristics of their literacy programs. 
The second part of the survey was divided 
into two subcategories: feedback on the 
tutor survey evaluation questionnaire and 
feedback on the student survey evaluation 
questionnaire. There were four questions in 
each subcategory, focusing on quality of the 
questionnaires (see Appendix for details). 



Results and Discussion (Findings) 

A total of 64 subjects returned the survey. 
The majority of ^e respondents were fe- 
males (87.1%), highly educated (85.3%), 
years of experience in adult literacy (six or 
more years, 72.7%), and Michigan-state 
certified (90.2%). Most of the agencies/ in- 
stitutions were either rural (40.7%) or urban 
(39.0%), and they senred a variety of ra- 
cial/ethnic groups. Detailed description of 
respondent demographics and their agency/ 
institution characteristics were displayed in 
Table 1 and Table 2, respectively. 



Data Analysis 

The respondents’ background and their 
feedback on the questionnaire were ana- 
lyzed first using descriptive statistics. The 
group difference in feedback was further 
examined using chi-square tests. More 
specifically, the statistical differences in the 
feedback on both questionnaires by Kind-of- 
Agency/Institution (rural, urban, other). 
Education (<college, college, graduate). 
Years in Adult Literacy (^ yr., 6-10 yr., >10 
yr.), Michigan CerWcation (elementary, 
secondary, other, no), and Gender (male, 
female) were examined using the chi- 
square tests, and type I error of the tests 
was set at 0.05. 
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Table 4 

Demographics of Respondents 



Variable 


Number I 


% 


Gender 






Male 


3 


87.1 


Female 


8 


12.9 


Highest Degree 

EariMKl:;... iv:, , 








3 


4.9 




5 


8.2 




23 


37.7 




25 


41.0 


DtioUHate 


2 


3.3 


Oil^ 


3 


4.9 


Years in Adult 

UtSTE^ 






:Leij» tfian 2 


3 


5.1 


2-5 


13 


22.0 


6-10 


26 


44.1 


llormore 


17 


28.8 


Ml State CertMcation 






Eiementeuy 


10 


19.6 


Secondary 


17 


33.3 


Vocational 


2 


3.9 


Combined 


9 


17.7 


Other 


5 


9.8 



Table 5 

Characteristics of Respondents’ 
Agencies/institutions 



Characteristic 


Number i 


% 


Kind of 

Agmicy/lnetitiition 

Rural 


24 


40.7 


Urban 


23 


39.0 


Suburban 


5 


8.5 


Other 


7 


11.9 


RaciatiEthnic 
ConH|)Qiiition 
Aftfceu^;Arnerk»h 
Asian/Pa:^ 
Caucasiar) 
Hispanic 
Native American 
Arab/Other 




Average Approximate 
Percentage 

24.61 

1.53 

65.81 

4.23 

2.07 

1.75 
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TUTOR/STUDENT QUESTIONNAIRE FEEDBACK 
STUDENT EVALUATION 



Overall, the feedback on the tutor evalua- 
tion questionnaire was very positive. The 
majority of respondents thought that the 
length of the questionnaire was “just righf 
(65.0%). Many of them edso tiiought that 
the “appropriateness of questions” of the 
questionnaire was “just righf (42.3%). Re- 
garding “clarity of the evaluation instru- 
ment,” most of them thought it was “just 
light.” Finally, many of them (39.6%) be- 
lieved that the questionnaire was applicable 
to their program(s). 

Still, the respondents pointed out that the 
evaluation questionnaire could be improved. 
A number of respondents believed that the 
instrument was “too long” (28%), the ques- 
tions were “not appropriate” (7.7%), the in- 
strument was “not deaf (11.1%), or the 
questionnaire was “not applicable” (8.35%). 
The detailed feedback on the tutor evalua- 
tion questionnaire is illustrated question-by- 
question in Figures 1, 2, 3, and 4, respec- 
tively. 



A similar feedback was found on the stu- 
dent evaluation questionnaire. Again, the 
overall feedback was positive. The majority 
of students believed that the length of the 
questionnaire was “just righf (65.8%), the 
questions were appropriate (52.6%), the 
questionnaire was clear (56.8%), and the 
questionnaire was applicable (52.6%). 

A small proportion of students also believed 
that the student evaluation questionnaire 
could be Improved, and thought that the in- 
strument was “too long” (34.2%), the ques- 
tions were “not appropriate” (26.3%), the 
instrument was “not deaf (10.8%), or the 
questionnaire was “not applicable” (5.3%). 
The detailed feedback on the student 
evaluation questionnaire is illustrated ques- 
tion-by question in Figures 5, 6, 7, and 8, 
respectively. 







Figure 1 : Feedback on Tutor Questionnaire 

Length of the Instrument 
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Figure 3: Feedback on Tutor Questionnaire 

Clarity of the Instrument 
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Figure 4: Feedback on Tutor Questionnaire 
Applicability of the Instrument 
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Figure 6: Feedback on Student Questionnaire 
Appropriateness of Questions 
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Figure 7: Feedback on Student Questionnaire 

Clarity of the Instrument 
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Figure 8: Feedback on Student Questionnaire 
Applicability of the Instrument 
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No statistical differences were found among 
groups in tenns of tiieir feedback on botii 
evaluation questionnaires. Chi*square sta- 
tistics by group were summarized in Table 3 



(Kind-of-Agency/lnstitution), Table 4 (Edu- 
cation), Table 5 (Years in Adult Literacy), 
Table 6 (Michigan Certification), and Table 
7 (Gender), respectively. 



Table 6 

Chi-Square Statistics of Feedback of Respondents 
on Kind— Agencies/Institutions 



1 OMPnCK V . 


Chi-Square 

Value 


dr 


P 












3.07 


4 


.55 


AppnHH^tmess 


5.69 


5 


.22 


Clarity 


6.26 


4 - 


.18 


A^licabiiity 


5.84 


4 


.21 


Studeirt Questiora^ 








Len^ 


2.31 


4 


.32 


Appropriateriess 


3.14 


4 


.54 


Clarity 


2.16 


4 


.71 


Applicability 


3.05 


5 


.55 
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Table 7 

Chi-Square Statistics of Feedback 
of Respondents on Educational Background 



FeecNbiaicik 


Ctii-Squarai 

Value 


df 


P 


Tirtor Ouesttonnaire 

.. ‘ - ^ : 


5.37 


6 


.50 




3.32 


6 


.77 




2.60 


6 


.86 


; ■ - 


4.69 


6 


.58 


■ ■ ’ "7 1 ^ y'-"-'' » 

- V- 


2.06 


2 
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3.98 


4 


.41 




4.20 


4 


.38 


Ajppiiic^ 


4.41 


4 


.39 



Table 8 

Chi-Square Statistics of Feedback 
of Respondents by Years in Adult Literacy 



FBtOb^ 


Chi^uare 

Yaiiie 


df 


P 


TuteyQuei^^ 








■ Lengrt ' 


0.82 


4 


.94 


Appriopdat^ess 


2.50 


4 


.64 


Clarity 


4.93 


4 


.29 


Applicability 


4.41 


4 


.35 


Student Quaetioiinaira 








Length 


0.78 


2 


.68 


Appropriateness 


3.55 


4 


.47 


Claris 


3.57 


4 


.47 


ApplicaUiity 


3.97 


4 


.41 
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Tables 

Chi-Square Statistics of Feedback of Respondents 
on Michigan Certification 



Feedback 


Chi^uare 

Value 


df 


P 


Tutor Que^onnaire 








Length 


7.18 


6 


.30 


Appropriateness 


8.14 


6 


.23 


Clarity 


3.32 


6 


.77 


Applic^iiity 


3.66 


6 


.72 


Stud^OuMdip^ 










1.74 


3 


.63 




3.74 


6 


.71 


ClarHy ■ 


4.93 


6 


.55 


Applicability 


4.42 


6 


.62 



Table 10 

Chi-Square Statistics of Feedback 
of Respondents by Gender 



Feedback 


Chi-Square 

Value 


df 


P 


Tutor Queetioniiaire 








length 


0.68 


2 


.71 


Appropriateness 


1.02 


2 


.60 


Clar% 


1.44 


2 


.49 


Applicat^ify 


0.98 


2 


.61 


Student Questionnaire 








Length 


0.11 


1 


.73 


Appropriateness 


0.26 


2 


.88 


Clarity 


0.75 


2 


.69 


Ap^icability 


0.76 


2 


.69 




5 'j 
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Conclusion 

Overall, the respondents’ feedback on botir 
questionnaires was positive. A small pro- 
portion of respondents believed that the 
questionnaires could be further improved. 
The respondents’ background and the 
characteristics of their agency had no im- 
pact on titeir feedback. 
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APPENDIX A 



Letter to Michigan Literacy Providers 
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December 7, 1994 

Dear Michigan Literacy Provider: 

Re; Proposed Survey hstruments for Statewide Application in Adult Literacy Programs 

The Detroit Literacy Coaiition (DLC) has just compieted phase one of a two-part series evaiuation modei 
pian to address aduit iiteracy program evaiuation needs. The objects is to create a user-friendiy evaiua- 
tion modei that can be appiied statewide to voiunteer iiteracy programs. 

At this time, we wouid iike to invite your participation in this process of improving the efficiency of our 
modei by compieting the feedback questionnaires enciosed. The data you provide wiii aiiow us to refine 
our instruments and deveiop effective, impartiai and user-friendiy toois that can be appiied to diveree 
popuiations around the state. We recognize, however, that differences wiii exist because of the criticai 
areas and concerns of so many different aduit iiteracy programs affecting disadvantaged sociai and eco- 
nomic groups such as disiocated workers, schooi drop-outs, drug rehabiiitation ciients, iiiiterate immi- 
grants, teenage mothers and prison popuiations in urban and rurai contexts. 

This is a great opportunity for you to participate in the creation of an eff^ive tooi. Remember that 
“evaiuation is not to prove but to improve.” Your cooperation and participation are greatiy appreciated. 
Compieted feedback questionnaires shouid be forwarded no later than 
December 30, 1994 to: 

Detroit Literacy Coaiition 
Wayne State University 
€001 Cass Ave., Room 451 
Detroit, Michigan 4202 
Tel: (313) 577-4612 (Crystal) or 
(313)577-3923 

Sincerely, 



Daphne W. Ntiri, Ph.D. 

Associate Prof. & Director, DLC 
Director, Project ALERT 

cc: Project Alert Team Members 

Gladys Coleman Walden 
Gregory Robinson 
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Glossary 

Adults Those people who have attained the legal and chronological status of adulthood, 
and are purposefully exploring the field of knowledge. In addition, these fields of 
knowledge are taking place in an organized group setting (Brookfield, 1991 , p. 

2 ). 

ALERT An acronym that stands for Adult Literacy Evaluation and Research T earn; the 
group that researched, analyzed and conducted this effort. 

Analysis Careful examination of a whole and its parts, including the relationships among 
them. 

Assessment A type of evaluation that is supposed to be based on a systern of value-free 

checking or ascertainment of the extent to which the worthwhileness, goals and 
objectives of a program are being met. A non-judgmental check to determine 
whether or not certain goals have been attained. 

DLC The Detroit Literacy Coalition, a community-based organization that meets the 

literacy needs of at-risk Detroiters through ttie collaborative networking of its 15 
member local agencies. 

Evaluation In general, research done to assess the worth of a program; also, the process of 
determining, the merit or value of something, or the product of that process. 

Feedback Information solicited from literacy providers after the primary survey is 

completed, in order to determine whether evaluation efforts have addressed the 
most critical concerns, and to identify ways in which improvements can be made. 

Formative An evaluation usually performed as ongoing programming is being 

evaluation conducted and/or during the developmental or improvement phase of the pro- 
gram. This process is designed to address operational procedures, and can be 
conducted by either the in-house staff or by an external evaluator. 

Illiteracy An inability to read, write, and comprehend short, simple sentences; however, to 
a greater extent this lack of basic skills may lead to an individual’s inability to ac- 
complish the kinds of basic and pervasive tasks necessary for everyday adult 
living (Lerche, 1985, p. 1). 

Literacy An individual’s ability to ret. write, speak, compute and solve problems at levels 
of proficiency necessary to ruiiction on the job and in society. 

Measurement (as it relates to literacy) is the process of determining the extent of related fac- 
tors, by comparison, estimation or judgment, and by reducing the data to num- 
bers that indicate strength or frequency of occurrence. 

PiioMest The procedures involved in pre-testing an item or tool in order to determine its 
effectiveness. 





Qualitative An evaluation that is more concerned with the quality of the program, and 
evaluation surveying information that cannot be measured (e.g., feelings, values, standards, 
etc.). 



Quantitative An evaluation concerned with amounts; the process of assessing 
evaluation information of things that can be measured exactly and objectively. 

Stakeholders Organized groups, indmduals, or other parties that have an interest politically or 
professionally, and that usually work for or reside within the geographical limits of 
the program. 



Sponsors The persons or organizations that initiate funds or provide the necessary re- 
sources used to conduct the evaluation. 

Summative An evaluation containing information offered only as a final or definitive 
evaluation judgment of the program’s effectiveness and used as the basis on which funding 
decisions can be made; usually conducted for the benefit of external audiences 
or decision-makers (e.g., funding agency, historian). 



Synthesis A drawing together of related items to form a coherent whole. 



TABE The test of adult basic education that is the most widely used standardized test, 

and is commonly used to determine program eligibility. 

Tutee Student of adult literacy programs who is being tutored. 

Tutor Usually a volunteer; however, all references to tutor in this document will apply 

equally to alternative funded persons working as tutors (e.g., work study stu- 
dent). 



Volunteer includes tutor and non-tutor or any person who works for no pay in order to help 
for Literacy fight against the problem of illiteracy. 



APPENDIX C 



Sample of Cover Letter Accompanying Questionnaire 



DETROIT LITERACY COALITION 
ADULT LITERACY EVALUATION RESEARCH TEAM 
(Project ALERT) 



Dear (Sample of cover letter accompanying questionnaire) 

The Detroit Literacy Coalition (DLC) is a 501 (c)3 nonprofit organization addressing the 
needs of literacy providers and adult learners. It is currently conducting a study on 
Adult Literacy Evaluation and Research in an attempt to design and develop a state- 
wide evaluation model. 

Your facility has been randomly chosen as a pilot site to pre-test the instrument and to 
solicit your input into the design of the instrument. We believe that “evaluation is not to 
prove, but to improve," so we encourage your cooperation in the development of an ef- 
fective, impartial evaluation tool. We will be contacting you by phone in the next few 
days to follow up on this correspondence. 

We thank you in advance for your cooperation and support. 

Sincerely, 



Gregory Robinson 
ALERT Coordinator 
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ALERT Feedback Form 
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DETROIT LITERACY COALITIOIV (DLC) 

WAYXE STATE EATVERSITY 

6001 Cass Avenue - Room 451 
Detroit, MI 48202 

Tel: (313) 577-4612 (Crjstal) (313) 964-1078 (Carla) FAX: (313) 577-8585 

Adult Literacy Evaluation and Research Team (ALERT) 

Feedback for Statewide Use 

Thi^ feedback fomi has been developed so that literacy stakeholders can provide feedback on the jutoraa d 
Student Questionnaires, enclosed. We encourage you to be as candid as possible. Feel tree 

to attach a^tioLl feedback, duplSte, and share the evaluation questionnaires and this feedback torm %vith 
colleagues and students. Please return you completed feedback forni-'s to DLC in the selt-addressed enclosed 
envelope by December 7, 1994. 



Name. 



Title 



Literacy Agency./lnstitution. 
Address, 



Cit\- 



Zip Code. 




1 . Is vour aaency/institution . . . 

l/Rurar ^ 2. Urban 3. Suburban 

2. Number of years in adult literacy... 

1. less than 2 2. 2 -5 



4. Other 



3. 6 - 10 



4. 1 1 - or more 



. <3 
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3. Highest dearee earned... _ 

1. High ichool 2. Associate 3. Bachelor 4. Master o. Doctorate 

4. Do you have Michigan state... , ^ ..-r *• 

1. Elementaiy Certification 2. Secondary Certification 3. \ ocational Certification 

4. Other 

5. Racial/etlmic composition of >'our clientele and approximate percentage... 

1. African American 2. Asian'Pacific Islander 3. Caucasian 

4. Hispanic/Latin 5. Native American 

6. Your gender... 

1. Female 2. Male 

GS 




Tutor and Student Evaluation Questionnaire Feedback Forms 

The following are feedback forms for the Tutor and Student Sun'ey Evaluation 
Questionnaires. 

Instructions: Circle the appropriate number. 

Tutor Siim'ey Evaluation Questionnaire Feedback Form 

1 . Length of the evaluation instrument: 

1. too short 2. too long 3. just right 

2. .Appropriateness of questions: 

1. not appropriate 2. adequate but could be improved 3. just right 

3. Clarit>- of the evaluation instrument: 

1. not clear 2. adequate but could be improved 3. just right 

4. Applicabilit>- of questiomiaire to >'our program population: 

1. not applicable 2. adequate but could be improved 3. just right 

Comments 



Student Siir\'ey Evaluation Questionnaire Feedback Form 

Personnel please assist students as necessary 

1 . Length of the evaluation instrument: 

1. too short 2. too long 3. just right 

2. .Appropriateness of questions: 

1. too difficult 2. adequate but could be impro\ed 3. just right 

3. Clarit\' of the e\'aluation instrument: 

1. not clear 2. adequate but could be impro\ ed 3. just right 

4. Applicabilin- of questionnaire to > our program population: 

1. not applicable 2. adequate but could be improved 3. just riglit 

Comments 



He appreciate your feedback, thank you. 
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Self-Evatuation Program 
Tutor Survey Questionnaire 



PARTI. 

IDENTIFICATION 



Ptease place an X in the boxas of all that apply. 



1.1 Area of program involvement 

□ a. tutor/instructor 

□ c. administration 

1.2 Nature of engagement in the program 

□ a. hill time 

□ c. volunteer 

1 .3 Utera <9 tutor/instnictor training 

□ a. Laubach Literacy intemti. (LLI) 

□ c. Eiementaiy Ml State Certification 

□ e. Mi State Reading Endorsement 



□ b. coordinator 

□ d. other 



□ b. part time 

□ d. paid 



□ b. Uteracy Volunteer of America 0.VA) 

□ d. Secondary Ml State Certification 

□ f. other^ 



1.4 Training and/or experience in aduh education 
□ a. yes □ p. no 



1.5 Length of lime with the program 

□ a. less than a year 

□ c. 3 > 5 years 

□ e. 11 or more 

1.6 Highest Qt^i^ree earned 

□ a. High School/GED 

□ c. Bachelor 

□ e. Doctorate 



□ b. 1 • 2 years 

□ d. 6 -10 years 



□ b. Associate 

□ d. Master 




( <. 
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PARTII. 

PROGRAM INFORMATION 



Please place an X in all the tioxes that apply. 



2.1 


Your student/s represent/s the following population/s 




□ a. rural 


□ b. semi-rural 




□ c. semi*urt>an 


□ d. prison/conrecUonal 




□ e. teenage mottters 


□ f. illitetate immigrants 




□ g. dislocated workers 

□ i. other 


□ h. drug rehabilHation 


2.2 


Primary radal/ethnic group you service 
□ a. African American 


□ b. Asian/Pacific Islander 




□ c. Caucasian 

□ Native American 


□ d. Hispanic 


2.3 


Instructional session length 
□ a. less than 60 minutes 


□ b. 60 minutes - 90 minutes 




□ c. 91 minutes- 120 minutes 

□ e. other 


□ d. more than 2 hours 


2.4 


Instructional session meeting day/s 
□ a. Monday 


□ b. Tuesday 




□ c. Wednesday 


□ d. Thursday 




□ e. Friday 

□ g. Sunday 


□. f. Saturday 


2.5 


Instructional session sr^eduiing 
□ a. once a week 


□ b. 2 times a week 




□ c. Stimesaweek 


□ d. 4 -5 times a week 




□ e. twice a month 


□ f. ottier 


2.5 


instrucKonal session day parts 

□ a. Morning 

□ c. Evening 


□ b. Afternoon (noon - 4:59) 


2.7 


Average number of students serviced per instructional session 




□ a.1 


□ b.2 




□ c.2-3 

□ e. 6 or more 


□ d.4-5 


2.8 


Instructional session location 
□ a. student’s home 


□ b. tutor's home 




□ c. library 


a d. literary center 




□ e. work 


□ f. churdt 




□ g. community center 


□ h. other 


2.9 


Materials used in your program 
□ a. print 


□ b. computer 




□ c. audio^cassette 

□ e. other 


□ d. video/films 




I O 



PARTS. 



, X in ttie nwnber bo* thil !• wort 



m 



BAP —OK 

1 2 



Gooa 

3 



VEHYOPOa 

4 



CYfMIgliT 

5 



Program Issues 
llllstsrisi for tutoring sessions 
Tutor training 



Support from the program staff or otWcs 
Student paiticipstlon in less on pisnning 
Student attendance 
Instnictional session iocsMon 



Homework assignments 
Discuss student's Nteraey goals 



3.9 j Student’s level of mottvstion 

3.10 1 Help student reach Hteracy goals 



1 8.11 I Studen t leferral to other support services 

1 3.12 1 Rank of overall program opemWon^ 




» 
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Self-Evaluation Program 
Student Survey Questionnaire 



1.2 



1.3 



1.4 
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PARTI. 

IDENTIFICATION 

Please answer the questions by putting an x in the box (El). 

^ PlMWContactyourtutorortaw^HyoMiiMdlMipiWMln^ 



1 .1 How long have you been in the program? 



a a. less than one month 
a b. 2-4 months 
a c. 5^ months 

Why did you start this program? 
a a. to get a job 
a b. please yourself 
a c. get a GED/High School Diploma 
a d. read the Bible 

What age group do you belong to? 

□ a. 16-18 

□ b. 19-21 

□ c. 22-25 

□ d. 26-30 

□ e. 31-35 

□ f.36^0 



a d. 7-12 months 
a e. more than 1 year 



a e. read to children 
a f. law/DSS 

□ g. learn to read and write 

□ h. other 



□ g. 41-45 

□ h. 46-50 

□ I. 51-60 

□ k. 66-70 

□ j. 61-65 

□ I. 71-plus 



What racial or ethnic group do you belong to? 
a a. African American □ d. Hispanic 

□ b. Asian/Pacific Islander □ e. Native American 

□ c. Caucasian 



1.5 Are you a parent? 
a a. yes 

1.6 Do you have a job? 
a a. yes 

1.7 What is your gender? 
a a. female 



a b. no 
□ b. no 



a b. male 

1 .8 What is the last grade you attended in school? 



r o 
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PART II. 

PROGRAM INFORMATION 

Please answer the questions by putting an x in the box (Bi). 

CS“ 

PlMM contact your tutor or tMchtr If you nMdhaiprMding. 



2.1 



How did you find out about the program? 



□ a. word of mouth 

□ b. poster/flyerAjrochure 

□ c. radio 

□ d. television 



□ e. church 

□ f. community center 

□ g. newspaper 

□ h. other 



2.2 How long are your classes? 

□ a less than 60 minutes □ d. more than 2 hours 

□ b. 60 minutes - 90 minutes □ e. other 

a c. 90 minutes - 120 minutes 



2.3 V\^at day/s of tiie week do you have your classes? 

□ a. Monday □ e. Friday 

□ b. Tuesday □ f. Saturday 

□ c. Wednesday □ g. Sunday 

a d. Thursday 

2.4 What part of the day do you have your classes? 

□ a. morning □ c. evening 

□ b. afternoon (noon - 4:59) 

2.5 How many times do you meet for class? 



□ a. once a week 


□ e. once a month 


□ b. two times a week 


a f. twice a month 


□ c. three times a week 

□ d. four - five times a week 


□ 0 . oftier 


Where are your class meetings? 


□ a. at home 


□ e. work 


□ b. tutor's home 


□ f. church 


□ c. library 


□ g. community center 


□ d. school 


□ h. other 
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2.7 How many students are there in your class? 

□ a. 1 myself □ d. 4-5 

□ b. 2 □ e. 6 or more 

□ c.3 

2.8 Do you and your tutor/teacher plan your classes together? 

a a. yes □ c. sometimes 

□ b. no 

2.9 V\^t materials do you use in class? 

□ a. print (books, paper) □ d. video 

□ b. computer □ e. other 

□ c. audio/cassette 

2.10 How many times have you stopped attending this program? 

□ a. 0 □ d. three times 

□ b. once □ f. four times or more 

□ c. twice 

2.1 1 If you were to drop out of this program what would be the most likely 
reason? 

a a. tutor/teacher □ g. weather 

□ b. feels like I am not learning □ h. no support 

□ c. location of classes □ i. work schedule 

□ d. time of classes □ j. family 

□ e. child care □ k. other 

□ f. transportation 

2.12 Would you attend a meeting for adult students, like yourself, to talk about 
learning? 

□ a. yes □ c. maybe 

□ b. no 
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PART III. 

ATTITUDES TO THE PROGRAM 

Please rank the following literacy attnudes issues. Place an X in the number box that is 

most in agreenMnt with your opinion 

BAD OK GOOD VERY GOOD EXCELLENT 

1 2 3 4 5 



Q. 


How Do You Feel? 


1 


2 


3 


4 


5 


3.1 


Your attendance 












3.2 


Your tutor’s/teacher’s attendance 












3.3 


Learning materials (books, workbooks, papers 












3.4 


Tutor’s/teacher’s help in reaching your reading goals 












3.5 


Program’s help with problems (refer to other agencies) 












3.6 


Home assignments 












3.7 


Pace of the class 












3.8 


Class meeting location 












3.9 


Understand your tutor/teacher 












3.10 


Life improved since beginning the program 












3.11 


Growth in your reading ability 












3.12 


Growth in your writing ability 












3.13 


Growth in your confidence since beginning the 
program 












3.14 


Improvement in your relationship/s with others since 
beginning the program 












3.15 


How you feel about yourself since beginning the 
program 












3.16 


Rate your classes 












3.17 


Rate your tutor 












3.18 


Rate the whole program 












3.19 


Recommend the program to others 
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APPENDIX F 



Self-Evaluation Program Questionnaires 



SELF-EVALUATION PROGRAM 
TUTOR SURVEY QUESTIONNAIRE 



nease check only one response in all tfie responses bdow unless stated otfierwise. 

1. nease indicate die ^jrpe of agency to be surveyed. 



2. Mease indicate your pontion in this program. 



3. Mease indicate the length of time you have been in this program. 

a. Less Aland months 

b. 6*12 months 

c. 12>18 months 

d. 18*24 months 

e. More than 2 years 

4. Mease indicate which of e following is the most effective source of information for knowledge about 
the center. 

a. Wordofmouth 

b. Poster or flyer 

c. Radio or television 

d. Referral 

e. Odier (specify) 

5. Mease indicate one mgjor reason for student enndling in the program. 

a. Driver's licmise 

b. Independent normal life 

c. Employment opportuidfy 

d. Lawr/govemment requirement 

e. Other (specify)_ 
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Pleaie indicate to what extent this program provides yon widi instructional materiaL 

a. To a great extent 



b. To a good extent 

c. To a moderate extent 

d. Toa little extent 

e. Odier(spedfy) 

7. Please indicate tbe average number of times that a student is absent from a session. 

a. Less than 2 times a month 

b. 2>3 times a mondi 

c. 4*5 times a mondi 

d. 6*7 times a month 

e. More dian 7 times a mondi 

8. nease i ndicate to what extent a student's interaction widi other students is beneficiaL 

a. To a great extent 

b. Toagood extent 

c. Toa moderate extent 

d. Toalitdeextent 

e. Notatall 

9. nease indicate to what extent your effort as an instructor plays a role in achieving students' goal. 

a. To a great extent 

b. Toagood extent 

c. To a moderate extent 

d. Toalitdeextent 

e. Notatall 



O 
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10. Please indicate to ndiat extent the program provides counsding and guidance to students. 

a. To a great extent 

b. To a good extent 

c. To a moderate extent 

d. To a little extent 

e. Not at all 

11. nease indicate to what metent your students do didr home as s ig nm ents on a regular basis. 

a. To a great extent 

b. To a good extent 

c. To a moderate extent 

d. To a little extent 

e. Not at all 

12. Please indicate how the students perceive the level of woilc a ss i gnm ents from you. 

a. Too hard 

b. Challenging 

c. Just right 

d. Easy 

e. Don’t know 

13. Please indicate die extent to vrhiefa student expectations are being met in the achievement of their 
goaL 

a. To a great extent 

b. To a good extent 

c. To a moderate extent 

d. To a little extent 

e. Notatall 



O 
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Heasc inilisate how die students perceive die pace of die program they are in. 

a. Too fast 

b. Fast 

c. Just right 

d. Slow 

e. Too slow 

Hease indicate in your opinion one nuyor barrier that pli^s a role in student absenteeism firom 
sessions. 

a. Baby*sitter problem 

b. Transportation 

c. Selfifamily health 

d. Fear of failure 

e. Other (specify) 

Please i ndica te to vdiat extent the learning experiences provided specifically address student goals. 

a. To a great extent 

b. To a good extent 

c. Toa moderate extent 

d. To a litde extent 

e. Not at all 

Please indicate to vdiat extent die program is beneficial to the students in the program. 

a. To a great extent 

b. To a good extent 

c. To a moderate extent 

d. To a litde extent 



e. Not at all 



18. Please indicate how you diink your students will rate die program they are in. 

a. Excellent 

b. Very good 

c. Good 

d. Average 

e. Not bad 

19. Please indicate if students recommend die program to odiers. 

a. Yes, without reservation 

b. Yes, widi some reservations 

c. No 

d. It depends (explain) 

20. Please make comments if you have aiqr. 
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SELF-EVALUATION PROGRAM 
STUDENT SURVEY QUESTIONNAIRE 



1. Please indicate die Idnd of program you are carreotly enrolled in. 



2. IHease indicate die purpose of your enrollment in fliis program. 



Please check only one response in aU the responses bdow unless stated otherwise. 

3. a. Less dian 6 mondis 

b. 6*12 mondis 

c. 12-18 months 

d. 18-24 months 

c. More than 2 years 

4. nease check ill sources that gave you information about this program. 

a. Word of mouth 

b. Poster or flyer 

c. Radio or tdevinon 

d. Referral 

e. Other (please specify) 

5. Please indicate one mi^r reason or goal for your enrollment in this program. 

a. Driver's license 

b. Independent normal life 

c. Employment opportunity 

d. Law/govemment requirement 

e. Other (please specify) 
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6. Please indicate to vdut extent this program provides you with instructional material. 

a. To a great extent 

b. To a good extent 

c. To a moderate extent 

d. To a little extent 

e. Notatall 

7. nease indicate die number of times you were absent from die program. 

a. Less than 2 times a month 

b. 2*3 times a month 

c. 4*5 times a month 

d. 6*7 times a month 

e. More than 7 timesa month 

8. Hease indicate to what extent die program allows you to interact with other students. 

a. To a great extent 

b. Toagood extent 

c. To a moderate extent 

d. To a iitde extent 

e. NotataU 

9. Please indicate to ^idiat extent your instructor's efforts play a role in achieving your goaL 

a. To a great extent 

b. To a good extent 

c. To a moderate extent 

d. To a Iitde extent 
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e. Not at all 



10. Please indicate to what extent dte program provides you connsding and guidance. 

a. To a great extent 

b. To a good extent 

c. To a moderate extent 

d. To a little extent 

e. Notatall 

11. Hease indicate to vdiat extent you have time to work on your as ri g nme nt at home. 

a. To a great extent 

b. To a good extent 

c. To a moderate extent 

d. To a little extent 

e. Not at all 

12. Hease indicate flielevd of worit assignments in this program. 

a. Too hard 

b. Challenging 

c. Just rigjit 

d. Easy 

e. Don't know 

13. Please indicate die extent to which your expectations arc being met in achieving your goals. 

a. To a great extent 

b. Toagoodextent 

c. To a moderate extent 

d. To a litde extent 

e. Not at all 



O 




r 

O i 



84 



Please rate die pace of the program you are in. 

a. Too fast 



b. Fast 

c. Jnst right 

d. Slow 

e. Too slow 

15. nease indicate gugmi^r barrier that plays a role in your class attendance. 

a. Baby*4dtter proUem 

b. Transportation 

c. SdFfamQy health 

d. Fear fsihire 

e. Other (please spedfy)^ 

16. Please indicate to what extent you think diis program will hdp you in achieving your goals. 

a. To a great extent 

b. To a good extent 

c. To a moderate extent 

d. To a little extent 

e. Notatall 

17. Please indicate to what extent your performance at home, at woric or at some other place improved 
since yon entered die program. 

a. To a great extent 

b. Toa good extent 

c. To a moderate extent 

d. To a litde extent 

e. NotataU 
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nease rate the prosram you are in. 

a. Excellent 

b. Very good 

c. Good 

d. Average 

e. Not bad 

19. Would you rec<Htnmend this program to others. 

a. Yes, without reservations 

b. Yes, with some reservations 

c. No 

d. It depends (please explain) 

20. Mease make comments if you have any. 
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POLICY STATEMENT 



The Detroit Literacy Coalition was created in 1985 to strengthen and enhance the 
awareness and co mmitm ent to literacy in the City of Detroit. With the partnership 
of business, labor, educational, public and private agencies, and the community- 
at-large, the Detroit Literacy Coalition aims to establish a unified, comprehensive 
and coordinated plan to address the illiteracy problem of Detroit 



OVERVIEW 



"The mission of the Detroit Literacy Coalition is to enrich the lives of individuals 
and families by helping them achieve their full potential tbrou^ literacy by sup- 
porting, promoting and ensuring the availability and accessibility of literaq^ serv- 
ices, thus contributing to a workforce that will ensure a strong economy and a 
promising future for the City of Detroit" 
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DLC Board of Directors 
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DETROIT LITERACY COALITION 



PRESIDENT 
Dr.ROSAMALLETT 
Wayne County RESA 



DIRECTOR 

Dr. DAPHNE WILLIAMS NTIRI 

Wayne State University 



STAFF 

PATRICIA TOMICH, Events Coordinator 
CARLA JORDAN, Clerical 
Wayne State University 



BOARD OF DIRECTORS 



Lt JOHN AUTRE Y 

Detroit Police Department 

VERNIT A BEVERLY 

WTVS-Channel 56 

RUTHBIERSDORF 

Detroit Public Library 

THOMAS E. COOK 

MEA/NEA Great Cities Program (Retired) 

GLORIA EDMONDS 

AKA Reading Experience Program 

EDITH EUBANKS 

Neighboiiiood Services 

ANTHONY HARRIS 

Graduate Student, Wayne State University 



PATRICK LINDSAY 

Chrysler Corporation 

GLORIA GRADY MILLS 

Michigan State Department of Education 

Dr. VERONA MORTON 

Highland Park Community College 

MARY ANN REILLY 

United Way 

Sr. MARIE SCHOENLEIN 

Dominican Literacy Project 

IDA SHORT 

Schoolcraft College 
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Detroit Literacy Coalition 

Mission Profile 



StateaiC Et of Purpose 



The purpose of die Detroit Literacy Coalition (DLC) is to promote fw flie total eradication of illitera^ in our com- 
munity and to advance die cause of reading and writing both for woric and pleasure. The DLC is committed to im- 
proving die quality of life of every Michigan citizen and resident by ensuring that increased access to basic education 
is available to diose deprived or diose vtiio for some reason or other missed out on earty educational opportumties. 
Its central tenet is diat total literacy is fundamental tc die well-being of Detroit society. 

♦ Fat the DLC, die ongoing re-evaluation of our values and perspectives of die world in which we 
live demonstrates die need to equip Detroiters widi more dian the basic literacy skills to prepare 
fw an increasingty tecbnolo^caliy advanced world diat requites sophisticated levels of knowledge 
for die w(»kplace. Functional illherates or "prisoners of sil«ice," according to John Kozol, live on 
die edge of society. The DLC recognizes diat die responsibility to provide for die marginalized 
sectors uho are constantly deprived tests on our volunteers, tutors and all who can help. 

♦ The DLC recognizes that diere is a "New Order" in die Detroit workplace. Whereas widi die old 
order fi mc rion al illitetates could bide behind dieir ignorance, in die new order die introduction of 
computers and the constantly changing standards and regulations fw industrial workers make it 
necessary fm* greater achievement in education of our employees. 

♦ The DLC as a pivotal fwce in literacy advocacy in Detroit will gain consistent, ongoing financial 
support diat will be distributed to member literacy providers. 

♦ The DLC is aware of die nontraditional femily structure diat exists today. Family and intergenem- 
tional literal area a part of die plan of die DLC. The emphasis on die fiimily, including parenting 
skills, is incorporated in die planning and execution of literacy programs around die city. 

♦ The DLC will be die communications and information clearinghouse for literacy material. 

♦ The DLC articulates two strong words, "trust" and "love" diat are lacking in the lives of many. As- 
sociated DLC providers are encouraged to offer programs that are situated in warm and friendly 
environments to help learners feel more comfortable. 
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♦ The DLC m ai ntains &e interest in jSlling in the gt^s in die existing literature to form a sound con- 
ceptual base fin* literacy research and advancement in Michigan and die nation. 

♦ The DLC's mission is primarify to serve the City of Detroit in generating city-wide awareness and 
coordinating adult literacy resources and services. The DLC has taken the leadership in meeting 
die needs of at-risk Detroiters dirough die collaboradve efforts of its members. As part of its mis- 
sion to e^qiand services and foster a heahhy alliance widi the associated literacy providers (Detroit 
Public Schools, Detroit Public Libraries, churches, prisons, fiatemides and chibs, community 
groiqis) it promotes public/private sector partnerships; it fosters greater corporate awareness of 
adult functional illiteracy; it wotics toward increasing business involvement in literaQ^ activities; it 
encourages volunteerism; it provides technical and networldng assistance; it promotes adult liter- 
acy services duough local and national media and dirough contacts wiA state agencies; and it 
stages highly visible activities such as die Literacy Summit and the Walkadion to boost die image 
of die organization and make it a household word. 

In short, die DLC has set die goal of alleviating illiteraQ^, that is, reducing the number of functional illiterates fiom 
200,000 to less dian 50% over the next few years in die metro Detroit area. This is particularly desirable for drop-out 
youth, special populations, and ethnic enclaves where illiteracy seems to predominate. Ultimately, our goal is to 
make die DLC an exenqilaiy model diat can be replicated elsewhere. 
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APPENDIX J 



Services and Primary Goals 



Services 

Special Events 
Publications 

Research 

Community Services 

Grantswriting and Fundraising 
Leadership Development 
Organizational Development 

Information Sharing 
Public Policy Strategies 

Dialoguing 
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Primary Goals 



To organize and present special events such as the Literacy Summit and 
the Wallcathon -diat increase awareness in the community about die 
scourge of illiteracy. 

To produce and circulate publications such as newsletters, directories 
and other information documents (legislative, administrative, curricular) 
that will keep literacy providers and the public informed and knowl* 
edgeable. 

To generate data bases and disseminate results of studies about literacy 
and education in general for metro Detroit, paying close attention to is* 
sues that wQl help alleviate the literacy dilemma. 

To offer to neighhoihood groups and literacy providers skills and tech- 
nical assistance such as program development and management, unor 
training, grant writing, board development, curriculum development, 
data collection, etc. 

To identify and obtain funds from a variety of sources that can make the 
DLC the central grant redistributing agency to promote literacy in the 
city. 

To be in a position to demonstrate leadership and to organize work- 
shops, coUoquia and symposia designed to promote leadership among 
literacy providers. 

To assist with organizational development of new and smaller groups 
that wish to join the literacy bandwagon. Such services often include: 
participant development, board development, curriculum development, 
marketing and grantwriting, promotion, and information sharing. 

To serve as the infom;<ta m clearinghouse for literacy materials. 

To increase public awareness of the problem of illiteracy with foil en- 
dorsement by the City of Detroit and foe Common Council. 

To promote exchanges, among Michigan's literacy providers such as foe 
voluntary literacy councils, adult basic education programs, state library 
systems, etc. 
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APPENDIX K 



List of Responding Literacy Sites 



Adult Basic Education 


Detroit Public Schools,- Adult Ed. 


Ishpeming, Negaunee, Nise Comm. Ed. 


Lura Bums 


101 Pioneer Ave. 


3700 Pulford 


Negaunee, Ml 49849 


Detroit. Ml 48207 


Adult Literacy Program 


Detroit PS Com. Based Ed East 


Janet Fulton 


Sharon Gogle 


Alpena County Library 


Harris School 


211 N. First Avenue 


3700 Pulford 


Alpena. Ml 49707 


Detroit. Ml 48207 


African Heritage Cultural Center 


Dominican Literacy 


PaMcia N<mi Gee 


Sister Mariene Lleder, OP 


'Detroit Public Schools 


9400 Counrille 


21511 W. McNichols 


Detroit. Ml 48224 


Detroit. Ml 48219 


Dominican Literacy Center 


Alger Area Literacy Council 


Donald J. BlllnskI 


Herbert M. Ingraham 


9400 Counrille 


R.R. 1, Box 1022 
Munising, Ml 49862 


Detroit. Mi 48224 




Dominican Literacy Center 


Allendale Community Education 


Donald Johnson 


Claire Voetberg 


9400 Counrille 


6633 Lake Michigan Drive 
Allendale, Mi 48401 


Detroit. Ml 48224 




Dominican Literacy Center 


Antrim-Kaikaska Literacy Council 


Nancy Mayer 


Toni Wayda 


9400 Counrille 


201 State Street 


Detroit. Ml 48224 


Manceiona, Ml 49659 


Dominican Literacy Project 


Brandi County Literacy Coundl 


Sister Marie Schoenlein 


Colleen Knight 


9400 Counrilie 


9501 E. Chicago 


Detroit. Ml 48224 


Coldwater, Mi 49036 


Down River Literacy Council 


Cheboygan County Libraries for Literacy 


Mary Lou Prevost 


Fred Brickley 


Southgate Adult Education 


Indian River Library, P.O. Box 160 


14101 Leroy 


Indian River, Mi 49749 


Southgate. Ml 48195 


Cheboygan County Libraries for Literacy 


Dukette Learning Center 


Cindy Lou Poquette 


Mary Jane Baumann, IHM 


3546 S. Straits Highway, P.O. Box 160 


530 W. Pierson Road 


Indian River, Ml 49749 


Flint. Mi 48505 
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Evart Community Education 
Carol Wdjcik 
161 1/2 N. Main Street 
Evart, Mi 49631 

Even Start Family Education Program 
Denise Schmitz-Bnking 
131 E. Aurora Street 
Ironwood, Ml 49938 

Gontcalm Adult Reading Council 
Ollivette Kassouni 
1401 Van Deiuse 
Greenville, Ml 48848 

Grand Blanc Community Education 
MarkR. Wallen 
G-1 1920 S. Saginaw 
Grand Blanc, Ml 48439 

Industrial Sites 
Barbara Hubert 
20018 Wisconsin 
Detroit, Ml 48221 

Kent County Literacy Council 

JudyZainea 

60 Library Plaza, N.E. 

Grand Rapids, Ml 49503 

Leelanau Literacy Council 
Hazel Le Anderson 
502 N. Traverse Lake Road 
Cedar, Ml 49621 

Linden Community Education 
7801 Silver Lake Road 
Linden, Ml 48451 

Literacy Council of Bay County 
Ann Gotten 
401 Banger St. 

Bay City, Ml 48706 



Literacy Council of Calhoun County 
Lynn Blake 
450 North Avenue 
Battle Creek, Ml 49017 

Literacy Council of Midland, Ml 
Dime H. Kau 

220 E. Main Street, Suite 206 
Midland, Ml 48640 

Literacy Network of Kalamazoo County 
Denise Duquette 
1000 E. Paterson 
Kalamazoo, Mi 49080 

Literacy Now! 

Verona Morton 

Highland Park Community College 
Glendale at Third 
Highland Park, Ml 48203 

Ludington Public Library Lit. Program 
Barbara Sutton 
217 E. Ludington 
Ludington, Ml 49431 

LVA - Detroit 
Heidi Hatcher 
330 Fisher Building 
Detroit, Ml 48202 

LVA ' Sanilac Council 
Martha Carpenter 
46 North Jackson 
Sanduski, OH 48422 

LVA ' Sanilac Council 
Grace Temple 
46 North Jackson 
Sandusky, OH 48471 

Ml Catholic Health Systems 
Beverly Clokajlo 
53 Chandler 

Highland Park, Ml 48203 
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Michigan Literacy Inc. 

Donna Audette 
P.O. Box 30007 
Lansing, Mi 48909 

Montcalm Adult Reading Council 
Virginia Schantz 
205 S. Franklin 
Greenville. Ml 48838 

Montcalm Adult Reading Council 
do Adult Education 
Olivette Kasscmni 
1401 Van Deinse 
Greenville, Ml 48838 

Mott Adult - Sarvis Center 
Kathryn Wiliams 
12037 Juniper Way, #530 
Grand Blanc, Mi 48439 

Oakland Literacy Council 
Wiiriam E. Engel 
17 S. Saginaw 
Pontiac, Mi 48342 

Oakland Literacy Council 
Nancy B. Geddes 
17 S. Saginaw 
Pontiac, Ml 48342 

Oakland Literacy Council 
Cadiryn Weiss 
17 S. Saginaw 
Pontiac, Ml 48342 

Osceola County Literary Council 
Marilyn Allen 
9890 Three Mile Road 
Sears, Ml 49679 

Project Lead - Adrian Pub. Library 
Janet Vem 
143 E. Monroe 
Adrian. Mi 49221 



Project Literacy 
Deborah Dei Zs^po 
P.O. Box 175 
Muskegan Oceana County 
Pentwater, Ml 49449 

R.E.A.D. Alpena County Library 
Susan Plowman 
21 1 N. First Avenue 
Alpena. Ml 49707 

Reading for Adults Literacy Program 

Doris E. Lance 

Alpena County Library 

211 N. First Avenue 

Alpena, Ml 49707 

Reggie McKenzie Foundation 

PatSmirii 

20 Bartlett 

Highland Park. Mi 48203 

Retired Senior Volunteer Program 
Tambrete D. Phillips 
P.O. Box 1025 
Gaylord. Ml 49735 

Roscommon County Literacy Council 
Jan Montei 
P.O. Box 320 
Roscommon, Ml 48653 

Schoolcraft College 
Linda Tolbert 
18600 Haggerty 
Livonia, Mi 48025 

Shiawassee Adult Literacy Association 
Maxine CapHan/Karen Fuoss 
515 W. Main Street 
Owosso, Ml 48867 

South Haven Area Literacy Council 

Kathy Stelfier 

600 Elkenburg 

South Haven, Ml 49090 
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South Haven Area Literacy Council 
Louise Wepfer 
600 Elkenburg 
South Haven, Ml 49090 

South Kent Adult Learning Center 
Lorraine Rozegnai 
319 Fairbanks 
Grand Rapids, Ml 49508 

South Kent Community Education 
Doris Hackett 
3529 S. Division Avenue 
Wyoming, MI 49548 

South Kent Comm. Ed. Basic. Lit. 
George Jasperse 
3121 Maple Villa SE 
Grand Rapids, Ml 49508 

Straits Area Community Adult Ed 
Ann Stafford 

840 Portage Road, P.O. Box 254 
St. Ignace, Ml 49781 



Tuscola Literacy Council 
Eiien Toner 
4117 Doerr Road 
Cass City, Mi 48867 

Volunteers for Adult Literacy 
Moiiie J. Hembruch 
Mott Adult High School 
1231 E. Kearsiey Street 
Flint, MI 48503 

Wayne County Adult Literacy Network 

Dr. Rosa Maiiett 

P.O. Box 807 

33500 Van Bom Road 

Wayne, Ml 48184-2497 

West Branch Rose City Adult Education 

Saiiy Campbeii 

960 S. M-33 

West Branch, Mi 48661 

WSU/Council for Excellence 
in Adult Learning 
2727 2nd Avenue, Suite 107 
Detroit, Ml 48202 
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